% Canoeboot Build System Audit 1
% Leah Rowe in Canoe Leah Mode™
% 9 June 2024

**A new release is now available, with these changes. Learn more by reading
about the [Canoeboot 20240612 release](canoeboot20240612.md).**

NOTE: This audit pertains to Canoeboot as of 9 June 2024 but the article is
published on 12 June 2024; nonetheless, the article date is set to June 9th.
For changes up to June 12th, please read the Canoeboot 20240612 announcement.

Introduction
------------

Canoeboot is a free/libre boot firmware project. It replaces your
proprietary BIOS/UEFI firmware, on supported x86 and ARM computers. It does
this by providing an automated build system to download, patch and compile
the various upstream sources (e.g. coreboot, GRUB, SeaBIOS). Coreboot is used
for hardware initialisation, configuring everything from your CPU, memory
controller all way to peripherals, readying the hardware so that it can run
software, e.g. GNU/Linux operating systems. You can essentially think of *cbmk*,
which is Canoeboot's build system, as a *source-based package manager*. It is
what the Canoeboot releases are built with. The *cbmk* build system essentially
implements
a *[coreboot distro](../docs/maintain/)*, the same way you might think of
a GNU/Linux distribution.

Extensive auditing has been performed on cbmk, since the Canoeboot 20240504
release. These audits fix bugs, reduce code bloat and generally improve the
efficiency of cbmk, adding and removing features in a careful, conservative
way, with a focus on *clean code*. Remember the magic words: code equals bugs.

This article covers changes from Canoeboot 20240504, up to
revision `4f6fbfde81f5176e5892d1c00627f8f680fd3780` from 9 June 2024.

Some notes about this audit
---------------------------

This is the *first* official Canoeboot audit. The initial Canoeboot releases,
from October/November 2023, incorporated changes from [Libreboot](https://libreboot.org/) Build System
Audit 3, and Canoeboot 20240504/20240510 included changed from [Libreboot](https://libreboot.org/) Build
System Audit 4, plus changes adapted from releases up to Libreboot 20240504 and
a bit beyond; however, Canoeboot used to be synced with Libreboot per each
Libreboot release - nowadays, it is synced *per commit* on both the build system
and the documentation, plus other repositories.

Therefore, whenever a Libreboot audit is announced, a corresponding Canoeboot
audit will also be announced. To keep it clean, we will simply refer to this
one as the *first* official Canoeboot Build System Audit, or *audit 1*.

ALSO: Canoeboot 20240510 was released *during* the audit; this changelog is in
reference to Canoeboot 20240504, *not* 20240510.

### Modest code size reduction

There are 1054 lines of shell script in the build system, versus 1208 in the
Canoeboot 20240504 release. Canoeboot's build system is written purely in
POSIX sh; not BASH, not KSH, not ZSH, jush sh!

This is a difference of 154 lines, or a 13% reduction. Despite the reduction,
numerous features have been added and a large number of bugs were fixed.

Summarised list of changes
--------------------------

Changes are in order per category, from newest to oldest:

### Feature changes

* **Download crossgcc tarballs as dependencies, when cloning coreboot.** We
  previously relied on the coreboot build system, which automatically fetches
  these when running `make crossgcc`, which we run automatically. However,
  the coreboot logic has to be patched for reliability because the GNU HTTP 302
  redirect often fails so we use a static mirror, and the logic has no
  redundancy. With this new change, we use the same tarballs but we specify
  two URLs, a main and a backup. This also means that the tarballs will once
  again be included in Canoeboot release archives, enabling offline builds.
* **Support downloading files as submodules, in Git repositories.** This
  complements the pre-existing feature where sub-repositories (Git) can be
  cloned into a subdirectory of a given main repo. We use this for crossgcc,
  as referenced above.
* New files under `config/dependencies/` for Fedora 40 and Ubuntu 24.04. Now
  you can run `./build dependencies fedora40`
  and `./build dependencies ubuntu2404` on each respective distro, to get
  the right build dependencies for building Canoeboot from cbmk.
* **NEVER** run `git submodule update`, *ever*. Instead, rely *solely* on
  config/submodule/ to define which dependencies should be downloaded, to each
  given subdirectory within a main project. This is using a feature described
  later on (in this audit report), whereby projects can have redundant
  submodule repositories defined; initially, this feature was an *override*
  where otherwise the submodule update command would be executed if
  the `.gitmodules` file existed for a given project; this override is now
  the *only* way to do it, and is thus the default behaviour. This may be
  considered a preventative bug fix, in case certain projects auto-download
  submodules that might cause us trouble in the future. It's better that we
  maintain tight control of submodules.
* **Summarising the next few changes mentioned below: out-of-source builds
  are now fully supported, for both single- and multi-tree projects.** (it was
  previously only supported on multi-tree projects)
* Moved builds of coreboot utilities (e.g. cbfstool) to `elf/utilname`,
  e.g. `elf/cbfstool/default/cbfstool` would be the new cbfstool binary location
  for the one build from coreboot in the `default` tree.
* script/trees: Now single-tree builds are skipped if a build exists
  under `elf/projectname/`, based on the presence of a `build.list` file; this
  is consistent with the same behaviour pre-existing for multi-tree projects.
* When building memtest86plus, the binary is now placed out-of-source,
  into `elf/memtest86plus`.
* When building flashprog, the binary is now placed out-of-source,
  into `elf/flashprog/`.
* Use new function `singletree` to decide whether to use submodules, rather
  than hardcoding a check for *coreboot* - NOTE: use of submodules was later
  disabled during this audit, replaced with custom handling in cbmk.
* For error exits caused by *improper commands* (as opposed to fault conditions
  while processing valid commands), don't directly call `err`; instead, call a
  newly written function `badcmd` which says that much, and links to the
  website (if `docs/` is present as in releases, it also points there).
* Added a `projectsite` file pointing to canoeboot.org, complementing the
  existing `projectname` file which contains the word `canoeboot`. This is
  used in the `version` command.
* **GRUB is now a multi-tree project.** Each given coreboot target can
  specify which GRUB tree it wants to use, each containing its own revision
  and patches, with its own GRUB configuration file. This can be used later on
  to provide specific optimisations on each given motherboard, but it is used
  at present to exclude xHCI patches on boards that don't need it; please also
  read the bugfix section (of this audit report) pertaining to this same topic,
  for more context. Before this change was implemented, all motherboards used
  the exact same GRUB revision, with the same patches and the same config.
* grub.cfg: scan `grub2/` last, on each given device/partition; this speeds
  up the boot time in most tests, because most setups use `grub/`,
  but `grub2/` is still used on legacy setups so we have to support it and, for
  reasons mentioned in the bullet point below, GRUB is very inefficient at
  generating the list of devices/partitions when using the `*` wildcard, so
  we can't scan `grub*/`.
* grub.cfg: it now scans a reduced set of devices/partitions by default, while
  still ensuring (in practise, on real systems) that all such devices and
  partitions will be scanned. We hardcode this, because the `*` wildcard in
  GRUB is *very slow* on some machines, due to the way the GRUB kernel
  constantly re-initialises the list of devices and partitions during operation.
  Scanning an *excessive* number of hardcoded device/partition numbers slows
  down the boot too, so this has been optimised. It has been tested and it
  shouldn't cause any issues on machines/setups that people actually use.
* **grub.cfg: scan distro-provided grub.cfg from ESP;** we previously only
  scanned the ESP for isolinux/syslinux configurations (which GRUB can parse).
* grub.cfg: Don't search for `*_grub.cfg` as this slows down the bootup
  sequence, and nobody really uses this anymore; Canoeboot's GRUB is much more
  robust these days, pretty much booting anything automatically, but you used
  to have to (regularly) use a `canoeboot_grub.cfg` file to override the default
  one provided by your distro. This legacy cruft has been removed, entirely!
* script/roms: Allow to override `grub_scan_disk` via `-s`, for
  example: `./build roms -s nvme t1650_12mb`
* **grub.cfg: Use `grub_scan_disk` to set boot order (rather, boot order by
  device type).** It is possible now to configure each motherboard with this
  variable, so that certain types of devices are scanned in a precise order;
  for example, scan NVMe SSDs first.
* **include/git.sh: Allow manual override of `git submodule` handling**, instead
  directly downloading Git repositories using `git clone`, into the subdirectory
  of a given main Git repository (as per `src/projectname` scheme). With this
  feature, it is possible now to specify a *backup* submodule repository, for
  redundancy, all while still allowing to reset the revision (and *patch* the
  given submodule). This has been used to provide greater redundancy when
  downloading coreboot submodules. It also allows to *limit* the number of
  submodules, so now we only download the ones we need, thus saving bandwidth
  especially during very large and long build sessions. - *NOTE: this was
  later changed so as to be the ONLY method for downloading submodules, skipping
  the actual git-submodule-update command entirely, on all projects.*
* **Native NVMe driver added to the GRUB payload**, allowing users to boot from
  NVMe SSDs where present on a given motherboard. The patch is courtesy of
  Mate Kukri, who ported SeaBIOS's own NVMe driver, converting all of the
  code to run properly within GRUB's own kernel. NVMe SSDs are now fully
  bootable on all machines that can have them, offering vastly superior
  read and write performance when compared to SATA SSDs.
* include/git.sh: Allow patching git submodules (NOTE: support for submodules
  was removed entirely, later in the audit, in favour of custom logic in cbmk
  for the downloading of such dependencies).
* Added Portuguese keyboard support in the GRUB payload (patch courtesy of
  the contributor by alias `samuraikid`).
* Removed all help commands, because it's just a duplication of documentation
  that is already included in releases anyway, and people using the Git
  repository require internet access anyway, so they can just use the website.
* Main build script: removed the functionality for generating source tarballs
  where the only source code included is U-Boot; we do not need this, because
  the larger source tarball containing all of Canoeboot also contains U-Boot.
* include/option.sh: Don't bother checking for GNU Tar, because we were only
  using it for reproducible tarball generation which didn't work yet anyway;
  there are still ways of doing it with BSD tar and so on.
* Print a two-line break before confirming the location of the generated
  release archive, when running release builds. This makes it more obvious
  to the operator.
* Removed all status checks from script/roms (formerly script/build/roms),
  because it's better to document this instead, and rely on testing regardless.

### Bug fixes

Some of these changes fix actual issues that were found in testing, while
others were fixed *before* being triggered/reported and are thus *preventative
bug fixes*. The logic in cbmk has been very intensively audited as is customary!

The changes are, from newest to earliest:

* script/trees: Exit with error status if a given project is not defined. It
  was previously decided that this script could be used to directly run Makefiles
  from any given directory, but this is no longer done as it was error-prone;
  this change prevents such usage, thereby preventing unstable conditions within
  the build system.
* **Create a lock file when running cbmk.** Only do it from the main parent
  instance, but not child instances of it; delete it at the end, after exiting
  from the parent process. If starting a separate parent process, that one
  will now immediately exit (with error status) if the lock file exists. This
  prevents the fault condition where the user accidentally runs the same cbmk
  instance twice, from the same work directory; it is only designed to be
  executed once, per work directory. This is similar to the locking feature
  you find in package managers such as apt-get. Also do this in release/
  directories, while building (but don't include a lock file inside the tarball).
* include/git.sh: When doing a global check for files in every project all at
  once, as defined by each respective (if existent) `nuke.list` file, hide
  the output. Only show the output when running it on a specific project, not
  the one in the for loop. This prevents user confusion / false bug reports.
* include/git.sh: Download coreboot as defined by `xtree` *before* downloading
  the main project that defined it, to prevent a situation where the main project
  is downloaded successfully but not the dependency (defined by `xtree`); this
  is to maintain the integrity of the build system under fault conditions.
* include/lib.sh: When a download fails (running the `download` function),
  don't then say that the file is "missing". Instead, actually say that the
  download failed, so that the operator has a better understanding.
* include/lib.sh: Hide stderr on the `download` function, for the initial
  check when verifying an existing file; although no problem existed on
  technical terms, the output was confusing because it made the user think
  there was a problem. The logic then downloads and re-verifies, and the
  output indicating *that* verification has not been hidden; if the file
  already exists, this is simply indicated by `e()`. This is considered a bug
  fix, because it fixes the bug where users made erroneous bug reports, by
  re-engineering the situation so that they do not make such erroneous reports.
  TL;DR hide a totally benign (non-)error message.
* include/git.sh: Provide better user feedback about what is being downloaded
  and where - although nothing was broken before, this lack of feedback was a
  bug because it made debugging harder. Provide more clarity for the user.
* include/git.sh: Download dependencies *before*, not *after*, downloading the
  project sources that depend on it. For example, pico-serprog depends on
  pico-sdk. If you were to download pico-sdk *after* pico-serprog, the latter
  may be downloaded and placed in src/, but then the former (sdk) could fail
  due to bad internet, and now the overall downloaded code is corrupt, and there
  was nothing checking for this after the fact; checking for it would be bloat.
  By downloading the dependency *before*, then if *that* download fails, so
  does the main one, and integrity is maintained within the build system.
* Preventative bugfix: don't check empty paths in `copy_elf` (of script/trees),
  even though this potential bug was not yet triggered. Play it safe.
* script/trees: Don't check pre-existing builds in elf/ if `build.list` is
  missing, otherwise it's too soon and builds are prevented in the first place;
  this was caused initially when supporting out-of-source builds for single-tree
  projects, as was already done on multi-tree. Now this is fixed.
* Documentation: only define the Untitled Static Site Generator
  in `config/git` - the dependencies (markdown files and images) are now
  defined in config/submodules/ instead. This prevents the bug where you could
  download one of the dependencies first which would make the main project,
  Untitled, un-downloadable, since the dependency projects go in subdirectories
  of the main project that depends on them.
* Handle serprog dependencies in config/submodule instead of relying on
  the git submodule update command, and only provide necessary modules. This
  prevents the bug where downloading a dependency first later prevented the
  main project from being downloaded, if the dependency was in a subdirectory
  of what depends on it.
* Build coreboot utilities on a number of threads as defined by `XBMK_THREADS`;
  although they already compiled, they would always do so on a single thread,
  which is considered a bug. Now they can be compiled on multiple threads.
* include/lib.sh: Don't use `./update trees -f` to build coreboot *utilities*,
  because it's quite error prone and not what that script is designed to do;
  it is only designed to operate based on strictly defined single- and
  multi-tree projects. Instead, call `make` directly.
* Don't use the presence of a `build.list` file to detect a multi-tree project
  when running `./update trees`; instead, check the presence of `target.cfg`
  down one level from `config/project/`, so: `config/project/*/target.cfg`
  instead of `config/project/target.cfg`. This way, if someone working on cbmk
  accidentally adds that `build.list` file in the wrong place, cbmk won't
  become unusable. This also means that single-tree projects can now provide
  a `build.list` file! (and some of them now do - look at the features section
  on this page)
* Move check for *root user* to include/lib.sh, *before* the version/versiondate
  files are written; these files need to be writeable by the standard user,
  otherwise cbmk will exit. If you run cbmk as root, except when running the
  dependencies command, it exits with error status; ironically, that very same
  check then prevented running as root-root, causing cbmk to become unusable
  until those files were either deleted or had ownership changed. This fix
  prevents the bug from occuring ever again, but people who were previously
  affected still have to fix these files (if they were written as root).
* Move dependency handling to include/lib.sh, *before* the version/versiondate
  files are written, and *exit* before they are written; this prevents writing
  the version/versiondate files as root, which previously occured when running
  a command such as `./build dependencies debian` (installs build dependencies
  from apt-get on a Debian machine). This bug ironically prevented cbmk from
  running at all, under such conditions, because the dependencies script
  required root, but cbmk exits with error status if running anything else as
  root, and if version/versiondate are owned by root, that prevents cbmk from
  running because writing to these files is the first thing it does, so an exit
  with error status would otherwise occur.
* config/git/: Bump to a newer revision of Untitled (static site generator),
  which thereby also imports the same fix as described in the next bullet
  point below, because Untitled had (and now no longer has) the exact same bug.
* include/lib.sh: check environmental variables properly, for example
  check that `${XBMK_RELEASE+x}` isn't unset; it was previously grepping
  the output of `set`, which led to a bug report by a user who had the
  variable `TMUX_TMPDIR` set, whereas `TMPDIR` was unset and cbmk was checking
  the latter; in this example, the bug caused cbmk to act as though `TMPDIR`
  was set, when it in fact wasn't, and code that used it then crashed because
  cbmk does `set -u -e` (and it does this precisely to catch such bugs like the
  one you're reading about now so that they can be fixed, like this one was!)
* **Re-configured GRUB so that none of the currently supported machines contain
  xHCI support**. This is a mitigation against the bug reported in [lbmk
  issue 216](https://codeberg.org/libreboot/lbmk/issues/216). This is done, by
  using the new *multi-tree* GRUB handling, which is mentioned above in
  in the section (of this audit report) pertaining to *feature changes*, whereby
  each motherboard can have its own GRUB revisions and patches, with its own
  GRUB configuration file (that could be uniquely optimised for it).
  We do not need xHCI patches on any Canoeboot machines, but the patches are
  free software regardless, and it's important to keep Canoeboot in sync with
  Libreboot. It may be that you want to enable it on a custom configuration,
  for example if you use a USB3 card on a KGPE-D16, but it is not currently
  enabled by default on any Canoeboot machines.
* **Fix vboot build issue when running cbmk in i686 (32-bit) host machines**.
  The patch, courtesy of *Luke T. Schumaker*, adapts vboot's vmlinuz extract
  function so that it uses pointer logic directly, instead of defining
  integers (of type `ssize_t`) which, the way it was written, caused GCC to
  believe that there would be a buffer overflow in code; the new code is more
  robust and should prevent such an issue. This is both an *acute* bug fix,
  fixing a bug that was actually triggered, and a preventative bug fix as the
  original code wasn't correct either, even on AMD64 hosts (where it happened
  to compile anyway).
* **GRUB: Never run it as a primary payload on any target but QEMU**. This is
  a preventative bug fix, after lbmk bug report issue 216:
  <https://codeberg.org/libreboot/lbmk/issues/216> - although it was caused by
  the xHCI patches, and only happened on Sandybridge hardware, and although
  this was later removed on those boards, GRUB is very complex and likely has
  a lot of memory corruption issues. SeaBIOS is more reliable, so: Canoeboot
  only provides *SeaBIOS* as primary payload, but allows you to execute GRUB
  from the SeaBIOS menu (the very same GRUB). Additionally: cbmk already
  supported a configuration whereby SeaBIOS reads a `bootorder` file in CBFS,
  making it try to run the GRUB payload first, while still allowing you to
  interrupt by pressing ESC to bring up an alternative boot select menu. This
  is now the *default*, on all x86 motherboards. This is a mitigation against
  future instability in GRUB because, if such issues happen again, it will not
  cause a brick since you can just use SeaBIOS instead, and skip booting to
  the GRUB payload (on the affected machines, BIOS GRUB still worked, which
  your distro provides and SeaBIOS executes it). *NOTE: GRUB was later made
  into a multi-tree project, with certain motherboards using a version that
  has the xHCI patches, if required, because the machines that actually need
  xHCI support were not affected by the bug referenced in issue 216.*
* Main build script: Check SUID before checking Git name/email, otherwise the
  version/versiondate files could be written as root and thus prevent building
  of cbmk, which (for most commands) is intentionally engineered to exit (with
  error status) if you run it as root.
* script/trees: Reset variable `makeargs` per target, so as to prevent
  pollution of this variable when switching from one build target to the next.
* script/trees: Added `UPDATED_SUBMODULES=1` to the make command when running
  any coreboot `make` command, to prevent coreboot from automatically fetching
  certain Git submodules; this is a preventative fix, fixed before it became
  a bug, which it likely would have become at some point as this is exactly
  what the coreboot build system does!
* Main build script: hide the output of `git init` when cbmk re-initialises the
  Git history, to prevent its output from being wrongly inserted into the
  output of commands such as `./build roms list` - such pollution would cause
  build errors, so it's important that the Git initialisation function either
  doesn't output anything, or that it should cause an *exit* if output is to be
  required.
* Added the `CHANGELOG` file to `.gitignore`. This means `./update release`
  will now work, on release archives, because cbmk re-initialises Git history
  when doing so, but the CHANGELOG file (when present) causes cbmk to skip
  all source downloads (which the release builder relies on).
* **Fix garbled output on 1440x900 monitors when using the Dell Latitude E6400.**
  The E6400 uses a reference clock (`DPLL_REF_SSCLK`) set to 100MHz, whereas
  libgfxinit assumed 96MHz. This timing descrepancy did not cause an issue on
  lower resolution displays, so we never caught it in earlier testing. Patch
  courtesy of Nicholas Chin, who debugged this issue alongside the user who
  reported it. It was fixed by making such timing configurable, within the
  coreboot build system, setting it to 100MHz on Dell Latitude E6400.
* script/roms: Skip a target when its config directory is missing, so that
  running a coreboot target with no configs in it will not yield an error;
  instead, it will now cause a non-error return.
* include/option.sh: If `.git` is missing, in a bare copy of cbmk (not a
  release archive), recreate the version/versiondate files manually so as to
  prevent a build error. Use of `cbmk.git` or the release archives is
  recommended, but some users directly download snapshots of `cbmk.git` from
  sites such as Codeberg, and there's no way for us to turn off this feature;
  even if we did, it may be present on other Git hosting sites, where users
  might host their own copy of cbmk.
* include/option.sh: Don't return non-zero status from the function
  named `mkrom_tarball`, because certain other functions rely on its return
  value to always be *zero*; instead, call `err` which will then yield
  an *exit* (with non-zero status). This means that the function will now
  always *return* zero, when it returns.
* include/git.sh: Remove `.git` directories *per-project*, as and when each
  project is being downloaded, instead of having it done all in bulk by the
  main build script. This kicks in when `XBMK_RELEASE` is set (release builds),
  to correct the over-use of disk space during such very large builds processes.
  This makes the build system less likely to OOM when running it inside tmpfs.
* Main build script: initialise Git history *before* running any command,
  because this is required for reliable use of the coreboot build system, which
  the *inject* command makes heavy use of. This reduces the number of errors,
  when running these commands from a release archive, where cbmk re-initialises
  a new Git history when you run it for the first time.
* Main build script: define `xp` as a global variable, to prevent it from
  being lost between functions.
* script/roms: Create full release tarball name, when generating releases.
* Main build script: exit (with error status) if not running directly from
  the root of the cbmk work directory.

### General code cleanup

In addition to *general* very sweeping code cleanup, condensing code lines
where possible and so on:

* include/lib.sh: Simplified the `download` function (used for crossgcc tarballs).
* include/lib.sh: Simplified the `singletree` function.
* include/git.sh: Simplified the `link_crossgcc` function.
* include/git.sh: Simplified the `nuke` function, because it was over-engineered
  to the extreme. Now it's more reasonable.
* include/lib.sh: Move download logic here from lbmk as of audit 5, for the
  feature where *files* can be downloaded as submodules, within Git repositories.
  Please read the notes about this in the *features* section.
* include/lib.sh: Shortened a string in the `e` function, so that the line
  does not exceed a length of 80 characters.
* include/git.sh: Unified the handling of git clone/reset/am commands into a
  single function, rather than duplicating the same logic across multiple
  functions.
* script/trees: simplify the `copy_elf` function; don't create the elf directory,
  create one defined by `dest_dir` instead (which is the elf directory with
  the subdirectory for that project concatenated). Only create it within
  the `copy_elf` function, which is only called if actually compiling the
  code. This avoids creating empty directories under elf/, for example under
  fault conditions.
* include/git.sh: Additional code cleanup, removing certain code that was in
  place because the code used to handle both `git submodule update` and the
  custom *override* logic for submodules; now only the override is used, at all
  times, so the code was cleaned up and optimised only for this.
* include/git.sh: Reduced code indentation in function `fetch_submodule`.
* include/git.sh: Renamed a few variables for increased code clarity.
* script/trees: Unified handling of coreboot `makeargs`.
* Moved function `handle_coreboot_utils` to script/trees (and renamed it
  to `check_coreboot_utils`), as it's only ever used from there.
* Moved cfgsdir/datadir variables to include/lib.sh, because it's also used
  from script/roms and script/trees; unify them under a common location.
* Handle `build.list` from config/data/, not config/ - this avoids needing to
  check for `build.list` in the `items` function on include/lib.sh, and it is
  now avoided.
* include/lib.sh: More user-friendly output from the `e` function, telling the
  user whether or not a file/directory exists. This is regularly used, for
  example when trying to download a project and the source code was already
  prepared.
* U-Boot on QEMU: removed the (currently) unused x86 target.
* grub.cfg: Split function `try_user_config` into multiple smaller functions.
* grub.cfg: Don't scan ESP on btrfs subvols as the ESP is always on a FAT32
  partition. This saves time during the bootup sequence.
* Renamed include/option.sh to include/lib.sh
* Main build script: simplified the logic for Git repository initialisation
  by *returning non-zero status*, instead of calling err, and handling this
  return status in the calling function.
* Main build script: condensed the logic for Git name/email checking into a
  simply for loop running `eval`, rather than having lots of separate but very
  similar Git commands.
* script/trees: Removed a few unused variables.
* include/git.sh: Moved logic for copying a Git repository to a new function.
* include/git.sh: Moved function `link_crossgcc` to a different location
  within the file, for proper top-down order of logic (required as per the
  cbmk coding style).
* include/git.sh: Split logic for crossgcc symlinking into its own function.
* include/git.sh: Skip submodule checks if `.gitmodules` missing (NOTE: later
  replaced with custom submodule handling in cbmk).
* include/git.sh: Merged `patch_submodules` in `prep_submodules` (NOTE: ditto
  to the same note below).
* include/git.sh: Split up submodule handling into a new function (NOTE: support
  for submodules was later replaced with custom logic in cbmk).
* include/git.sh: Shortened a few variable names.
* include/git.sh: Removed redundant check for the existence of the patches
  directory, when patching a given project. This is unnecessary, where it was
  removed, because the patching function itself also checks this. Reduction
  in code size by *one line*.
* include/git.sh: Removed function `fetch_from_upstream` and merged its logic
  into calling function `fetch_project_trees`, the only calling function, since
  the logic in `fetch_from_upstream` was very small and splitting made no sense.
* include/option.sh: Renamed `mktar_release` to `mkrom_tarball`.
* script/roms: Renamed function `moverom` to `copyrom`, because it runs `cp`,
  not `mv`, therefore is is *copying* a file, not moving it.
* script/roms: Simplified the logic for listing available serprog build targets.
* script/roms: General simplification of configuration handling for payloads.
* Main build script: removed the `excmd` function and merged its logic into
  the `main` function, and then `main` was cleaned up significantly.
* Main build script: don't make `script_path` a global variable; this allowed
  a reduction in code size by precisely *one line of code*.
* Main build script: merged the functionality of function `check_git` into
  the `main` function, then deleted function `check_git` (which was in
  the file include/option.sh).
* Main build script: general simplification of the logic handling source code
  downloads in function `fetch_trees`.
* Main build script: Use `UTC+0000` when initialising git repository commit
  dates (for initial commits).
* Removed the `check_project` function, and placed its logic directly
  inside `include/option.sh` so that it automatically runs in every script
  that sources it.
* Main build script: General cleanup on the code handling file deletions
  under function `fetch_trees`.
* Main build script: delete function `mkversion` and, in its calling function,
  simply print the string contained in variable `relname`.
* Main build script: general cleanup on the logic that handles tarballs.
* Main build script: Remove `mkrom_images`, and move its logic into the only
  calling function within that same file.
* include/option.sh: Removed the function `insert_version_files` and merged
  its logic into its only calling function.
* Unified all logic for handling SHA512 checksums, placing it inside
  include/option.sh for use elsewhere.
* Move image tarball generation to script/roms (formerly script/build/roms).
* Removed redundant function `extract_ref` from include/mrc.sh
* Removed an errant comment from include/git.sh
* Switched to a one-level directory structure for main scripts, rather than
  two-level; for example, script/build/roms is now script/roms
* Merged script/update/release into the main build script
* Merged script/build/serprog into script/build/roms
* script/build/roms: remove unnecessary command (errant return)
* Merged include/err.sh with include/option.sh, into include/option.sh
* script/build/roms: fixed improper use of variable outside a function
* build/build/roms: more reliable exit status in `skip_board()`
* script/build/roms: split up `main()` into multiple smaller functions

Revision updates
----------------

Some revisions were updated as part of standard routine, but happened to be
done during this audit. Those updates are as follows:

### SeaBIOS

Bump SeaBIOS to revision `e5f2e4c69643bc3cd385306a9e5d29e11578148c`, which has
these changes relative to the old one:

```
* e5f2e4c6 pciinit: don't misalign large BARs
* 731c88d5 stdvgaio: Only read/write one color palette entry at a time
* c5a361c0 stdvga: Add stdvga_set_vertical_size() helper function
* 22c91412 stdvga: Rename stdvga_get_vde() to stdvga_get_vertical_size()
* 549463db stdvga: Rename stdvga_set_scan_lines() to stdvga_set_character_height()
* c67914ac stdvga: Rename stdvga_set_text_block_specifier() to stdvga_set_font_location()
* aa94925d stdvga: Rework stdvga palette index paging interface functions
* 8de51a5a stdvga: Rename stdvga_toggle_intensity() to stdvga_set_palette_blinking()
* 96c7781f stdvga: Add comments to interface functions in stdvga.c
* 2996819f stdvga: Rename CGA palette functions
* 91368088 stdvgamodes: Improve naming of dac palette tables
* 70f43981 stdvgamodes: No need to store pelmask in vga_modes[]
* 1588fd14 vgasrc: Rename vgahw_get_linesize() to vgahw_minimum_linelength()
* d73e18bb vgasrc: Use curmode_g instead of vmode_g when mode is the current video mode
* 192e23b7 vbe: implement function 09h (get/set palette data)
* 3722c21d vgasrc: round up save/restore size
* 5d87ff25 vbe: Add VBE 2.0+ OemData field to struct vbe_info
* 163fd9f0 fix smbios blob length overflow
* 82faf1d5 Add LBA 64bit support for reads beyond 2TB.
* 3f082f38 Add AHCI Power ON + ICC_ACTIVE into port setup code
* 3ae88886 esp-scsi: terminate DMA transfer when ESP data transfer completes
* a6ed6b70 limit address space used for pci devices.
```

### Flashprog

Updated to revision 5b4fdd1 from 2 May 2024, rebasing the MX workaround patch.

This imports upstream changes, relative to the previous revision:

```
* 5b4fdd1 z60_flashprog.rules: Add udev rule for CH347
* 72c9e40 meson: Check for CPU families with known raw mem access
* 3458220 platform/meson: Port pciutils/pci.h workaround to Meson
* f279762 platform/meson: Check for libi386 on NetBSD
* 14da5f7 README: Convert to Markdown
* 8ddea57 README: Document branching and release policy
* 2522456 util/list_yet_unsupported_chips.sh: Fix path
* cbf9c11 spi: Don't cross 16MiB boundaries with long writes
* 823a704 dediprog: Skip warning on first attempt to read device string
* e8463c8 dediprog: Revise prefix check for given programmer id
* 38af1a1 dediprog: Revise id matching
* 4661e7c amd_spi100: Use flashprog_read_chunked() for progress reporting
* cdcfda2 read_memmapped: Use flashprog_read_chunked() for progress reporting
* 7679b5c spi25: Replace spi_read_chunked() with more abstract version
* ca1c7fd spi25: Normalize parameters of spi_nbyte_read()
* e36e3dc dediprog: Use default_spi_write_256
* 522a86d linux_spi: Use default_spi_read()/_write_256()
* 806509b cli_classic: Turn progress reporting into a progress bar
* 842d678 libflashrom: Return progress state to the library user
* aa714dd flashprog.c: Let select_erase_functions() return byte count
* 2eed4cf serprog: Add SPI Mode and CS Mode commands
* 821a085 dediprog: Implement id reading for SF600 and later
* 274e655 dediprog: Read device string early
* 0057822 dediprog: Add protocol detection for SF700 & SF600Plus-G2
* fb176d2 dediprog: Use more general 4BA write mode for newer protocols
* 0ab5c3d dediprog: Split device type and version parsing
* bdef5c2 dediprog: Use unsigned conversions to parse device string
* 5262e29 dediprog: Try to request 32B device string (instead of 16B)
* e76e21f dediprog: Get rid of some unnecessary hex constants
* 5a09d1e udelay: Lower the sleep vs delay threshold
* 03ad4a4 linux_mtd: Provide no-op delay implementation
* 211c6ec serprog: Refine flushing before synchronization
* 383b7fe serprog: Test synchronicity before trying to synchronize
* d7318ea serprog: Move synchronicity test into separate function
* 9a11cbf Let the flash context directly point to the used master
* aabb3e0 writeprotect: Hook wp functions into the chip driver
* 89569d6 memory_mapped: Reduce `decode_sizes` to a single `max_rom_decode`
* 929d2e1 internal: Pass programmer context down into chipset enables
* 7c717c3 internal: Pass programmer context down into board enables
* e3a2688 Pass programmer context to programmer->init()
* 2b66ad9 Start implementing struct flashprog_programmer
* 4517e92 memory_bus: Drop stale `size == 0` workaround and FIXME
* b197402 memory_bus: Split register mapping into own function
* 0e76d99 memory_bus: Move (un)map_flash_region into par master
* 9eec407 Perform default mapping only for respective chips
* 56b53dd wbsio_spi: Request memory mapping locally
* 5596190 it87spi: Request memory mapping locally
* 46449b4 spi25: Drop stale `bus == SPI` guards
* ab6b18f spi25: Move 4BA preparations into spi_prepare_4ba() hook
* 901fb95 Add prepare/finish_access() hooks for chip drivers
* a96aaa3 dediprog: Support long writes of 16MiB and more
* 1338936 Consider 4BA support when filtering erase functions
* 8d36db6 flashprog.8: Fix up serprog example
* d2ac303 flashprog.8: document new serprog cs parameter
* d1b9153 chipset_enable.c: Add Genoa to mendocino entry
```

As a reminder:

Canoeboot now uses Flashprog instead of Flashrom; Flashprog is a fork of
Flashrom, lead by Nico Huber after a dispute with the new leadership of
Flashrom, and it was felt that Flashprog is a better choice for Canoeboot.

Git log
-------

This entire set of changelogs is based on the precise Git history in cbmk,
relative to Canoeboot 20240504 which is from where the audit began.

The latest changes are listed first, going all the way down to earlier changes:

```
* 4f6fbfde81 minor code cleanup in the build system 
* 070aee6728 re-add ability to use cbfs grub.cfg as default 
* b4acd0f73c trees: exit with error if project undefined 
* fd9664c567 build: also make a lock file during release build 
* 686bad6d4e lib.sh: more useful lock message 
* f1caf89a28 create a lock file during builds 
* b6dc23bc67 git.sh: hide e() output on for loop 
* e51eae0d25 lib.sh: fix regression 
* 8b1a54d19e git.sh: download xtree *before*, not after 
* 14bba2d789 git.sh: fix deletion path in nuke() 
* ab4c4d406f lib.sh: less confusing error in download() 
* 2eaaa63f58 lib.sh: hide stderr on download() 
* 9e2584fbd9 lib.sh: simplify download() 
* 79fb79d239 lib.sh: fix redundancy in download() 
* e8b1d45631 lib.sh: simplify singletree() 
* 90a8ef90b0 git.sh: further simplify nuke() 
* c6b692208b git.sh: simplify link_crossgcc() 
* c043e5810d git.sh: simplify nuke() 
* 323a17d0c8 Add dependency scripts for Fedora 40 and Ubuntu 24.04 
* 62b2310a28 add crossgcc tarballs to config/submodules/ 
* 8a34a0d338 git.sh: support downloading *files* as submodules 
* 0730513709 git.sh: remove unnecessary line break 
* ad05266f8d import file download function from lbmk c202dc61 
* b8e9eab0ba lib.sh: shorten a string in e() 
* a29cf274bc git.sh: fix submodule path 
* 7ac2264f53 git.sh: simplify prep_submodules() 
* 7c8173ebd4 git.sh: unified handling of git clone/reset/am 
* 573199c07d trees: simplified copy_elf() handling 
* d0d9b1204f git.sh: simplify submodule handling 
* df5d7c18bf git.sh: provide feedback for repository downloads 
* 591c7d28e0 git.sh: download "depend" projects *before* 
* 548d1e20c1 git.sh: reduced indentation in fetch_submodule 
* 12a04e8de2 git.sh: reduced indentation in prep_submodules 
* 9825e97a83 git.sh: *never* run git submodule update 
* 860deb3e7e lib.sh: rename variable for clarity 
* 8d5edd4f06 trees: don't check empty path in copy_elf() 
* c1176bbd28 trees: fix build issue caused by bad elf check 
* c88fb8c129 trees: fix listfile check in copy_elf() 
* 9168d33741 trees: don't say check elf/ if build.list missing 
* db09530905 trees: don't do elfcheck if build.list missing 
* 99418a7e82 define mdfiles/images in config/submodules/docs/ 
* 83d84797d8 libopencm3 to config/submodules/ on stm32-vserprog 
* c3cabcddf9 add tinyusb to config/submodule/ for pico-sdk 
* e4eb82e089 trees: unified coreboot makeargs 
* f7170092c8 trees: use multiple threads to build cbutils 
* 1d7a6f04c9 move handle_coreboot_utils to script/trees 
* ff16d27991 put coreboot utils in elf/, not cbutils/ 
* 3748f710c9 fix build issue building coreboot utils 
* a30bfd334f trees: skip single-tree build if a build exists 
* b682b4ddca use correct memtest86plus path in script/roms 
* 4749a5a29f put memtest86plus builds in elf/memtest86plus/ 
* 0e9d9b33b2 put flashprog builds in elf/flashprog/ 
* 7fe0106fa0 trees: also print "DONE! check elf/dir" on single 
* 74759d876a trees: handle build-test on multi-tree projects 
* 98e9cf6864 git.sh: use singletree() to decide submodules 
* b3b887567a remove cbcfgsdir variable (unused) 
* cb446e7d24 move cfgsdir/datadir variables to lib.sh 
* 7d99786a1a handle build.list from config/data/, not config/ 
* a61794dfca don't use build.list to detect multi-tree projects 
* 878056f37b move id check to lib.sh too 
* 3900642471 move root check to lib.sh (bugfix) 
* 740b1803fa bugfix: move dependencies handling to lib.sh 
* 4e25e335ed bump untitled revision again 
* 44ef38b335 bump untitled revision in git config 
* 7b9431e336 lib.sh bugfix: check environmental variables right 
* 2478252f67 lib.sh: more friendly output from e() 
* d21fd016ac badcmd: don't print "no context given" 
* 663de3bab4 badcmd: link directly to the maintenance manual 
* 1d866d17d8 better help text on invalid commands 
* 1204bc3c96 build: print the project website address on help 
* ca0e9354f6 add projectsite file: point to canoeboot.org 
* eb4ac3c334 make GRUB multi-tree and re-add xhci patches 
* 347a104ae6 u-boot on qemu: remove currently unused x86 target 
* 23e66c113d grub.cfg: scan /boot/grub.cfg last 
* 6151316b91 grub.cfg: scan grub2/ last 
* 36b3be95cf grub.cfg: search a reduced list of devs/partitions 
* 71a17efc06 grub.cfg: scan grub.cfg from ESP 
* 8bc7e3a539 grub.cfg: split up try_user_config 
* cb4bacc9d9 grub.cfg: don't search for *_grub.cfg 
* ea7e6e1659 grub.cfg: remove unnecessary path for isolinux 
* 1beca3b781 grub.cfg: don't scan EFI on btrfs subvols 
* 0662519cca Fix building vboot on i686 
* 224dce632b git.sh: do not remove .submodules 
* a36504aa31 delete u-boot test/lib/strlcat.c using nuke() 
* cdce8ba70b make nuke function more generic 
* 2c1f6f5e7a do not allow dashes in coreboot target names 
* 7dc5d35929 roms: allow user override of grub_scan_disk 
* bcb65846d3 grub.cfg: actually support setting boot order 
* 2887b77ae4 trees: use CPUS=x on regular coreboot make 
* a056583762 update gitignore 
* 1ac4f7409e roms: fix bad eval when comparing options 
* 724dbfe0ce grub.cfg: add spdx header 
* 66f5faac73 re-configure grub_scan_disk on various targets 
* bb92776943 remove grub_scan_disk in all target.cfg files 
* 935447b035 grub.cfg: use grub_scan_disk to set boot order 
* 75b6fbf302 GRUB: remove XHCI patches for now (will re-add) 
* 07340d9711 minor correction 
* 9f489b43d5 roms: make grubfirst if seabios_withgrub=y 
* fca9b19e18 coreboot: only run GRUB as a secondary payload 
* b75490f8fc flashprog: bump to 5b4fdd1 from 2 May 2024 
* d147c5d915 rename include/option.sh to include/lib.sh 
* f534b0e973 merge nuke() back into git.sh 
* a02b152f44 rename nukeblobs to a more generic name 
* cb1918c5d7 roms: remove errant reference 
* 4cff3c7d33 roms: rename bstr variable 
* dc487df12f git.sh: remove errant whitespace 
* cbb2f4f8a9 general code cleanup in the build system 
* 583135e548 build: simplify git_init() 
* aaff90f5a5 build: do root check before git check 
* 687fdacc78 build: simplify git checks 
* 84ee6a1ed8 option.sh: fix bad check for version/versiondate 
* 3554593fd8 trees: reset makeargs per target/project 
* b09261a901 trees: also use UPDATED_SUBMODULES=1 on crossgcc 
* 698548ac59 trees: add UPDATED_SUBMODULES to coreboot make 
* c8c516703f trees: write -C on the make command first not last 
* aa15eef32f config: add backup coreboot submodule repositories 
* 9e88ef2449 coreboot/default: remove chromeec from module.list 
* 27f21c32d3 git.sh: break if a submodule clone succeeds 
* 38fca598fb coreboot: only download the necessary submodules 
* b5aa8b2d35 git.sh: allow finer control of git submodules 
* 9339c6f3fd build: hide git-init output 
* 31e089aff3 option.sh: generate version file if .git not found 
* 7ec023907b update/trees: remove unused variable 
* 2b0e71412e git.sh: move repo copying to a new function 
* d71c4d326e git.sh: move link_crossgcc to end of file 
* 0d7c249c9b move deblob function to new file "deblob.sh" 
* 1300f09e67 git.sh: move xgcc linking to a new function 
* 24934e6569 git.sh: don't include --checkout in submodules 
* 5e0129eb0f git.sh: skip submodules if .gitmodules missing 
* 7f82622caf git.sh: merge patch_submodules in prep_submodules 
* 9c0a7f14fc git.sh: split submodule handling to new function 
* b593127795 git.sh: remove errant line break 
* 19f694bf2a git.sh: remove another meaningless check 
* 71a9fcced8 git.sh: shorter variable names 
* 6693588857 git.sh: remove meaningless check 
* 5c459ad4ac git.sh: remove variable not meaningfully used 
* 7be7bb8edb add CHANGELOG to .gitignore 
* 3b2ebda890 Fix E6400 display reference clock patches 
* 995f052bb0 fix building coreboot images on i686 hosts 
* 31d2c818eb Also try unlocking encrypted volume on NVMe 
* 58f6741fb4 git.sh: fix invalid command in git_prep() 
* f58b01c300 Add NVMe support to GRUB2 payload 
* b892036edf Fix E6400 display issue with 1440 x 900 panel 
* f81c7ed8e9 Add pt qwerty keymap to lbmk 
* 849466c0ac git.sh: allow patching submodules 
* 8d4d063ace git.sh: don't delete .git if src/project/project 
* 0ecb062df0 build/roms: skip target if config/ dir missing 
* 4783c5b90e more minor cleanup in the build system 
* 10ecf41ee0 git.sh: remove fetch_from_upstream() 
* ddcb793bd2 option.sh: don't return 1 in mkrom_tarball 
* ae8637b620 option.sh: mktar_release to mkrom_tarball 
* 309c3b1f33 build/roms: rename moverom to copyrom 
* a39c95cfac minor code cleanup in the build system 
* f102e21ab6 build/roms: simplify serprog list command 
* 7a565c9f43 build/roms: simplified config payload checks 
* a243dc2308 option.sh: err if config directory is missing 
* c28166ff9e option.sh: print the config filename being checked 
* 9fd504e24a git.sh: Remove .git if XBMK_RELEASE=y 
* e4956478db build: remove initcmd() and simplify main() 
* f2b3bb142d build: initialise git first (before commands) 
* 571932d33e build: remove excmd() and simplify main() 
* 525f5525d3 build: don't make script_path a global variable 
* fbac2d8fe6 Implemented failsafe options at boot and inside menus for enabling/disabling serial, spkmodem and gfxterm 
* 3e5db248dd cbmk: allow easier sync with lbmk 
* e71189420f remove help commands (user should read docs) 
* 23854de888 option.sh: delete check_git() 
* 2c5f52ce29 build: define "xp" in the global variables 
* 48c5c57cff build: simplify for loop in fetch_trees() 
* c2baebc79a build: simplified downloads in fetch_trees() 
* 18d0e53480 ./build release: don't do u-boot-only archives 
* d8a923f766 build: use utc+0 when initialising git repo dates 
* 0794127986 remove check_project() (always set variables) 
* c8bc797f31 build: simplify deletions in fetch_trees() 
* 363ec7512c build: delete mkversion() (just print relname) 
* ae44676727 build/roms: clean up tarball handling 
* 3469836f18 rm src/u-boot/*/test/lib/strlcat.c in u-boot 
* c57dfefe91 build: remove mkrom_images 
* 6ab8c2c446 build: use same tarball name on uboot-only release 
* 21436c6a8f build/roms: create full release tarball name 
* 90c528032b option.sh: don't bother checking for GNU tar 
* 422d36a07c option.sh: remove insert_version_files() 
* ca1806f20e cleanup: remove mkvdir 
* a0ea7f7a92 unified sha512sum creation for tarballs 
* 09fcc343a3 move rom tarball creation to script/roms 
* 5c888669c6 disable x301 for next release (for now) 
* 91c90d763f print two line breaks before confirming release 
* d423421995 remove all status checks. only handle release. 
* 4826364afb git.sh: remove errant comment 
* 541430016f move script/*/* to script/ 
* 9084ab15ab build: print usage for special commands 
* f12c2f284f merge script/update/release into build 
* 41f4ee3c2d Canoeboot 20240510 release 
* 0580373ff9 bump seabios to e5f2e4c69643bc3cd385306a9e5d29e11578148c 
* 17b5cb2749 further modify the README (stragglers) 
* 628e91a3b9 build: further prevent non-cbmk-work-directory 
* e761a494c8 build: exit if not running from cbmk directory 
* eb8a02e808 build/roms: print serprog help 
* a398011180 merge script/build/serprog with script/build/roms 
* cd5c2573ac build/roms: remove unnecessary command 
* da748de455 merge include/err.sh with include/option.sh 
* 3acac46536 err.sh: correct copyright info 
* 6bdbb70dbc build/roms: don't rely on x in handle_target 
* 1c84d0fc9d build/roms: don't use exit status from skip_board 
* 0ada63b629 build/roms: split up main() 
* 5cecd9e394 build/roms: allow searching status by mismatch 
* 97d502ccc8 tone the README way, way down 
```

This is 206 changes, since Canoeboot 20240504.
